Papers by Maartje Ter Hoeve
Analyzing Dialectical Biases in LLMs for Knowledge and Reasoning Benchmarks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Previous work has shown degraded performance of large language models for under-represented English dialects. |
| Approach: | They analyze the effects of typifying “standard” American English language questions as non-”standard” dialectal variants on multiple choice questions. |
| Outcome: | The results show that typifying “standard” American English language questions as non-”standard” dialectal variants can reduce performance 20% . |
Assessing the Role of Data Quality in Training Bilingual Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that adding more languages can degrade performance for some languages while improving others. |
| Approach: | They propose a data filtering strategy to select high-quality bilingual training data with only high quality English data. |
| Outcome: | The proposed approach improves bilingual model performance by 2–4% and reduces bilingual models performance gaps to 1%. |
Discriminating Form and Meaning in Multilingual Models with Minimal-Pair ABX Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that multilingual models encode languagespecific information and language-agnostic features, but the nature and interaction of these representations is not fully understood. |
| Approach: | They propose a set of training-free ABX-style discrimination tasks to evaluate how multilingual language models represent language identity (form) and semantic content (meaning). |
| Outcome: | The proposed tasks show that language discrimination declines over training and strengthens over time and stabilizes in deeper layers. |
mRAKL: Multilingual Retrieval-Augmented Knowledge Graph Construction for Low-Resourced Languages (2025.findings-acl)
Copied to clipboard
| Challenge: | Knowledge Graphs are structured multirelational graphs that store factual knowledge. |
| Approach: | They introduce a Retrieval-Augmented Generation (mRAKL) based system to perform mKGC. |
| Outcome: | The proposed approach improves over a no-context setting with an idealized retrieval system. |
Analyzing the Effect of Linguistic Similarity on Cross-Lingual Transfer: Tasks and Experimental Setups Matter (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior work on cross-lingual transfer often focuses on a small set of languages from a few language families and/or a single task. |
| Approach: | They analyze cross-lingual transfer for 263 languages from a wide variety of language families . they include three popular NLP tasks: POS tagging, dependency parsing, topic classification . |
| Outcome: | The proposed approach is based on linguistic similarity measures for 263 languages . the results show that the effect of linguistic similarities on transfer performance depends on a range of factors . |
Training Bilingual LMs with Data Constraints in the Targeted Language (2025.findings-acl)
Copied to clipboard
| Challenge: | a large number of languages have insufficient data for pretraining, but most non-English models are trained on scrapes of the web. |
| Approach: | They propose to use data from an auxiliary language to boost model performance . they quantify the performance gap between training with data in a data-rich auxiliary and training in the target language . |
| Outcome: | The proposed method boosts model performance in a target language with insufficient data . it also explores the benefits of translation systems and the limitations of model scaling when data is limited. |
How Value Induction Reshapes LLM Behavior (2026.findings-acl)
Copied to clipboard
| Challenge: | Induction of values into LLMs can have unintended effects on the user interacting with it. |
| Approach: | They investigate the unintended effects of value incorporation into models by fine-tuning existing preference datasets and measuring their effect on safety, anthropomorphism and QA benchmarks. |
| Outcome: | The proposed model improves safety, anthropomorphism and QA benchmarks by inducing values and incorporating values into the model. |
GrammaMT: Improving Machine Translation with Grammar-Informed In-Context Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Experiments show that GrammaMT enhances translation performance on open-source instruction-tuned LLMs for various low- to high-resource languages across three benchmarks: (1) largest corpus, (2) challenging 2023 SIGMORPHON Shared Task data, (3) even in an out-of-domain setting with FLORES. |
| Approach: | They propose a grammatically-aware prompting approach that uses Interlinear Glossed Text . they propose gloss-shot, chain-gloss and model-glooss prompting strategies that are training-free . |
| Outcome: | Experiments show that GrammaMT improves translation performance on open-source instruction-tuned LLMs for various low- to high-resource languages across three benchmarks. |
What Makes a Good and Useful Summary? Incorporating Users in Automatic Summarization Research (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing research on automatic text summarization does not fully align with students’ needs. |
| Approach: | They propose a survey methodology that can be used to investigate the needs of users of automatically generated summaries. |
| Outcome: | The proposed method can be easily adjusted to investigate different user groups. |
On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization (2024.findings-emnlp)
Copied to clipboard
Yong Lin, Skyler Seto, Maartje Ter Hoeve, Katherine Metcalf, Barry-John Theobald, Xuan Wang, Yizhe Zhang, Chen Huang, Tong Zhang
| Challenge: | Reinforcement Learning from Human Feedback (RLHF) is an effective approach for aligning language models to human preferences. |
| Approach: | They compare the accuracy of DPORM and EXRM with a reward function for scoring human preferences. |
| Outcome: | The proposed methods can approximate an EXRM on the limit infinite samples, but it is unclear how effective they are in practice. |